Networks of Belief

Belief networks / Bayesian networks

Introduction In modeling real world tasks, one inevitably has to deal with uncertainty. This uncertainty is due to the fact that many facts are unknown and or simply ignored and summarized. Suppose that one morning you find out that your grass is wet. Is it due to rain, or is it due to the sprinkler? If there is no other information, you can only talk in terms of probabilities. In a probabilist...

Augmentable Gamma Belief Networks

To infer multilayer deep representations of high-dimensional discrete and nonnegative real vectors, we propose an augmentable gamma belief network (GBN) that factorizes each of its hidden layers into the product of a sparse connection weight matrix and the nonnegative real hidden units of the next layer. The GBN’s hidden layers are jointly trained with an upward-downward Gibbs sampler that solv...

Belief Networks Revisited

Entropy and Belief Networks

The product expansion of conditional prob­ abilities for belief nets is not maximum en­ tropy. This appears to deny a desirable kind of assurance for the model. However, a kind of guarantee that is almost as strong as max­ imum entropy can be derived. Surprisingly, a variant model also exhibits the guarantee, and for many cases obtains a higher perfor­ mance score than the product expansion.

Parallel Learning of Belief Networks

Learning belief networks from a large dataset over a large domain can be computationally expensive even with a single-link lookahead search. It has been shown that a class of probabilistic domain models cannot be learned correctly by several existing algorithms which employ a single-link lookahead search. When a multi-link lookahead search is used, the computational complexity of the learning a...